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A  MODEL  FOR  ORDINAL  FILTERING  OF  DIGITAL  IMAGES 
By:  M.  F.  JANOWITZ 

§1.  Introduction .  A  monochromatic  digital  image  may  be  thought  of 
as  a  finite  rectangular  array  of  numbers.  Each  number  in  the  array  repre¬ 
sents  some  sort  of  average  value  for  a  specified  type  of  radiation  from  a 
fixed  geoqraphic  or  geometric  region,  with  adjacent  numbers  corresponding 
to  adjacent  regions.  The  digital  image  is  subsequently  transferred  to  an 
actual  picture  by  means  of  some  sort  of  video  display  device.  Ar  will 
soon  be  seen,  however,  a  given  digital  image  can  produce  more  than  one 
picture,  so  one  must  examine  the  meaning  of  the  display  of  this  type  of 
data.  This  question  will  be  examined  within  the  context  of  a  model  that 
allows  one  to  classify  various  filter  techniques,  and  to  compare  com¬ 
peting  techniques  so  as  to  decide  which  is  better.  In  that  the  under¬ 
lying  model  is  order  theoretic  in  nature,  it  is  appropriate  at  this  point 
to  present  some  needed  background  material  from  the  theory  of  partially 
ordered  sets.  Though  the  material  can  be  presented  in  a  more  abstract 
fashion,  we  choose  to  stay  within  the  image  processing  framework  in  which 
it  will  be  appl ied. 

In  what  follows,  X  will  denote  a  finite  rectangular  array  of  points, 
with  X  having  m  rows  and  n  columns.  The  point  in  the  i—  row  and 
j—  column  is  denoted  x(i,j)  or  simply  x  if  it  is  not  important  to 
specify  the  exact  coordinates.  A  monochromatic  digital  image  may  then 
be  thought  of  as  a  mapping  d  :  X  - >  L  where  L  denotes  a  finite 
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chain.  Though  the  exact  nature  of  L  may  vary  from  computer  to  computer. 

a  typical  representat ion  may  he  obtained  by  taking  I.  -  1 0, 1 . 2!>5  1 

where  i  ^  j  in  L  has  its  usual  meaning  in  the  real  number  system. 

For  M  a  nonempty  subset  of  L,  we  agree  to  let  vM  and  aM  denote 
respectively  the  largest  and  smallest  elements  of  M.  The  symbol  p' 
or  P'(X)  will  be  used  to  denote  the  collection  of  nonempty  subsets  of 
X,  partially  ordered  by  set  inclusion.  In  connection  with  this,  the 
symbols  c,  u,  n  will  have  their  usual  set  theoretic  meanings. 

Now  let  d  :  X  L  be  any  mapping.  Notice  that  d  may  be 

uniquely  extended  to  a  join  homomorphism  D*:P'(X) - ->  L  by  ■ he 

rule 

D*(Y)  =  v{d(y)  :  y  e  Y}. 


The  pair  (D*,D)  of  mappings  has  a  number  of  interesting  features  in 


that 
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(Rl)  D*  is  isotone 

(R2)  D  is  isotone 

(R3)  Y  c  D[D*{ Y ) ]  for  ail  Y  P'(X) 

(R4)  h  >  D*[D(h) ]  for  all  h  >  hd. 

This  says  that  D *:P‘  - >  [h^  ,255]  is  residuated  in  the  srnse  t  '  ’. 

p.  11.  The  reader  is  referred  to  that  source  for  a  development  of  t>e 
properties  of  residuated  and  residual  mappings.  What  we  shall  np.-g 
the  fact  that  we  have  established  one-one  correspondences  between  three 
classes  of  objects: 

(A)  mappings  d  :  X  - ->  L 

(B)  residuated  mappings  D*:P’(X)  - >  [hrf ,255] 

where  h^  =  A{d(x)  :  x  e  X} 

(C)  residual  mappings  D  :  [h^ , 255]  - >  P’(X). 

The  link  from  (B)  to  (A)  is  accomplished  by  observing  that  d  is  the 
restriction  of  D*  to  singleton  subsets  of  X  with  the  remaining  links 
arising  from  the  theory  of  residuated  mappings  [1]. 

Suppose  now  that  9  :  L  - >  L  is  isotone.  If  M  =  [e(0),?55]  = 

{h  :  h  >  o(0)},  it  is  easy  to  see  that  if  0  is  regarded  as  a  mapping 
from  L  into  M,  then  6  is  residuated  with  0+  :  M  ■>  L  g^ven  by 

0+(k)  =  v{h  :  0(h)  <  k} . 

This  may  be  illustrated  by  defining  0  on  {0,1,2,3,4,5,6,71  by  means 
of  the  table 


Then  M  =  {3, 4, 5, 6, 7}  and  e'  is  given  by 


h 

3 

4 

5 

6 

1- 

7 

0+(h) 

1 

2 

3 

7 

i 

7 

. 

There  is  one  final  needed  item.  If  d:X  - •>  L  and  P  :L - >  L 

is  isotone,  then  Od : X  - >  L  with  o  °  D*  its  associated  residuated 

mapping.  As  in  the  proof  of  [2],  Lemma  7.1,  p.  68, 

(0  o  D*)+  =  0  o  e+. 

In  summary  then,  a  picture  may  either  be  viewed  in  the  form  (A), 

(B)  or  (C)  as  previously  described.  Somewhat  closer  contact  may  be 
made  with  the  model  described  in  [2]  by  working  with  the  lattice  of 
all  subsets  of  X,  P(X).  In  a  manner  similar  to  that  described  earlier 
in  the  section,  one  establishes  a  bijection  between  the  following  classes 
of  objects: 

(A)  mappings  d:X  - >  L 

(B)  residuated  mappings  D*:P(X)  - ->  [hj.255] 

where  h^  is  some  fixed  lower  bound  for  { d ( x )  :  x  e  X} 

(C)  residual  mappings  D:  [h^ ,255]  - >  P(X) 

The  situation  will  be  examined  in  more  detail  in  the  next  section. 


>  L  be  a  monochromatic 


§2.  Display  of  digital  images.  Let  d:X  - 

digital  image,  where  L  might  be  {0,1  , . . . ,255} .  The  digital  image  d 
is  converted  to  an  actual  picture  by  means  of  some  sort  of  visua1  display 
device.  The  display  device  either  produces  a  black  and  white  picture 
by  associating  with  each  integer  i  in  L  an  intensity  of  grey  C[i], 
or  it  produces  a  color  picture  by  associating  with  i  a  color  C[ i 1 . 

In  the  latter  situation,  the  display  is  controlled  by  a  color  lookup 
table  of  the  form 


C[0]  C[l]  C [2 ] 


C[255J 


where  the  256  colors  are  chosen  from  a  palette  consisting  of  2 
possible  colors  with  k  representing  the  number  of  bits  assigned  to 
represent  a  given  color.  The  colors  in  the  table  are  then  linearly 
ordered  by  the  rule 


C[i ]  <  C[j]  if  i  <  j  in  L. 


For  purposes  of  this  discussion,  it  will  be  assumed  that  the  colors 
or  grey  shades  C[0],  C[1 ] , . . . ,C[255]  are  fixed.  Many  image  pro¬ 

cessing  systems  come  equipped  with  a  means  of  dynamically  modifying  the 
lookup  table  so  as  to  enhance  certain  portions  of  the  picture.  This 
deserves  a  careful  consideration. 

We  are  given  a  digital  picture  d  where 


a 
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If  0  is  an  isotone  mapping  on  L,  a  new  picture  may  be  create-1  by 
looking  at 


P  d  :  X  - •>  L. 


But  this  involves  applying  0  to  each  and  every  pixel  x  €  X  to  change 
the  color  representation  from 


x  - ->  d ( x )  - >  C[d(x)] 


x  - ■>  6d(x)  - ■>  C[0d ( x ) ]  . 


The  same  effect  may  be  achieved  by  changing  the  lookup  table  so  that  it 
becomes 


c[e(o)]  I  c[e(i)]  I  c[e(2)] 


C[0 ( 2  55 ) j 


Since  this  only  involves  changing  256  entries,  it  is  the  preferred 
approach.  This,  however,  has  dramatic  implications.  When  a  digital 
image  is  entered  into  the  computer,  one  can  obtain  not  just  a  single 
picture  but  a  whole  class  of  pictures  that  arise  by  means  of  lookup 
table  modifications.  Tnis  should  cause  one  to  examine  the  significance 
of  the  actual  data  values  in  a  given  picture  d. 

An  alternate  view  of  a  digital  picture  involves  associating  with 
each  color  C[h]  that  portion  of  the  picture  that  is  painted  with  the 


color  C[h].  Thus  one  would  look  at 


instead  of 


x - d(x) - ••  C[d(x)]  . 

What  happens  to  the  alternate  representation  when  an  isotone  mapping 
6  is  applied  to  d?  As  long  as  the  restriction  of  6  to  the  range  of 
d  is  injective,  this  is  easy  to  answer,  since  one  simply  looks  at 

C[0(h)]  - ->  h  - >  {x  :  d(x)  =  h}. 

What  happens  though  if  0  merges  distinct  levels  of  the  picture?  One 
can  certainly  still  consider 

C[0(h)]  - •>  {x  :  0d(x)  =  0(h)), 

but  it  turns  out  to  be  more  useful  to  do  a  second  alternate  representation 
by  means  of 


C[h]  - >  {x  :  d ( x )  £  h}. 

Since  for  h  >_  h^  =  A{d(x)  :  x  e  X}  it  is  true  that 
8d(x)  <  h  <  —->  d(x)_<  e+(h) , 

we  then  have 

C[h]  - >  0+(h)  - •>  {x  :  d(x)  <_  0+(h)}  . 

The  second  alternate  representation  associates  with  each  color  C[h]  those 
pixels  x  that  are  painted  with  C[h]  or  some  color  that  preceded  it  in 
the  lookup  table. 


§3.  Filters.  It  is  often  desirable  to  take  a  picture  d  :  X - -•>  L 

and  enhance  it  to  produce  a  picture  F(d)  :  X  - •>  L.  This  is  done  by 

applying  some  sort  of  spatial  or  frequency  domain  filter  to  d.  To  achieve 
the  most  generality,  let  us  agree  to  call  any  transformation  of  a  pic¬ 
ture  d  :  X  - ->  L  to  a  picture  F(d)  :  X  - >  L  a  filter.  It  is  under¬ 

stood,  however,  that  for  a  filter  to  be  useful,  it  must  somehow  or  other 
aid  in  the  understanding  of  the  original  picture.  At  any  rate,  applying 
a  filter  to  d  produces  F(d).  This  may  be  illustrated  schemati  :al ly  by 


We  now  apply  an  isotone  mapping  0  to  the  lookup  table.  This  transforms 
both  the  original  and  filtered  versions  of  the  picture  as  indicated 
bel ow : 


Suppose  now  that  an  object  is  observed  in  0F(d)  that  was  not  observed 
in  any  of  the  other  images.  Is  it  real  or  is  it  an  artifact  of  F  and/or 
0?  Unless  one  understands  the  relation  between  9d  and  0F(d),  this  Is 
difficult  to  answer.  An  ideal  relationship  would  be  if 


since  this  says  that  the  process  that  produced  F(d)  from  d  also 
produced  0F(d)  form  0(d).  This  now  leads  to  a  natural  question. 

For  a  given  class  of  isotone  mappings  0  :  L  - >  L,  what  filters  F 

have  the  property  that 

F(9d)  =  0F(d) 

for  all  digital  images  d  :X - ->  L?  In  connection  with  this,  it  will 

often  be  useful  to  view  a  digital  image  d  :  X  - >  L  as  a  residual 

map  D  :  M  - •>  P'(X)  where  M  =  [hd ,255]  and  hd  =  *{d(x)  :  x  ■-  X}. 

The  filter 

d  - •>  F(d) 

then  takes  the  form 

D  - ■>  F(D) 

where  F(D)  is  the  residual  mapping  associated  with  F(d).  Notice 
then  that  by  [2],  Lemma  7.1,  p.  60, 

F ( 0d )  =  0F(d)  < >  F(0  o  0+)  =  F(D)  o  P+  . 

Definition,  (i)  Let  M  c  L  and  e  :  L  - >  L  be  isotone.  The 

filter  F  is  said  to  be  (e,M)  -  compatible  in  case 

F(9d)  =  0F(d) 

for  all  images  d  with  range  contained  in  M. 

(ii)  The  filter  F  is  said  to  be  monotone  equivariant  if 
it  is  (u ,M)  compatible  for  all  o  such  that  oL  is  injective. 


&  .*V  r*  <• 


k* 


k-- 


p 
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Our  first  goal  will  be  an  examination  of  the  meaning  of  monotone 
equivariance  for  filters  F.  It  should  be  mentioned  that  though  this  has 
essentially  already  been  done  in  [3],  it  will  still  be  useful  to  repeat 
the  proofs  here. 


Lemma  1 .  If  F  is  monotone  equivariant,  then  range  F(d)  -  ran 


for  every  d 

:  X  - ■>  L 

Proof:  If 

h  e  range 

d. 

one  can  define  6:  L  - •>  L  so  that  e(k)  =  k 

for  all  k  c 

range  d. 

but 

0(h)  5 *  h.  Then  d  =  ed,  whence 

F(d)  =  F(0d)  =  0F(d). 

If  k  =  (Fd) (x) ,  then 

k  =  ( Fd) (x)  =  0[( Fd) (x)]  =  e(k). 

In  that  h  ^  0(h)  we  conclude  that  h  i  range  F(d). 

Theorem  2.  If  F  is  monotone  equivariant  and  if  d  has  range 
hQ  <  hi  c  ...  <(■>£,  then  F(d)  depends  only  upon  the  sequence 

D(h0)  c:  D(h1)  c  ...  c  D(ht) 

and  is  independent  of  the  actual  levels  h^  <  h-|  <  ...  <  h^. 


Proof.  Let  d  :  X  - >  L  be  as  above,  and  let  Y .  =  D(h. )(i  =  0,. . . I , . . .  ,t) 

In  view  of  Lemma  1,  the  output  is  determined  by  the  sequence 

(FD)(h0)  c  (FD)(h])  ■  ...  c  (FD)(ht) 


since  h.  £  h  <  implies  (FD)(h)  =  ( FD) ( h ^ ) -  To  show  that 

the  output  is  independent  of  the  actual  levels  h . ,  choose  a  seqjence 

k0  <  k]  <  ...  <  kt 

of  elements  of  L  and  define 

c  :  X  - >  L 

so  that  Y .  =  C(ki)-  IT  0  :  L  - •>  L  is  defined  so  that  e( hi )  =  k. 

one  then  has 

0d  =  c , 

whence 

F(c)  =  F(-'d)  =  eF(d). 

Then 

F(C)  =  (FD)  o  o+ 
and 

(FC)(k.)  =  (FD)(0+(ki))  =  (FD)(hi). 

This  shows  a  monotone  equivariant  filter  to  be  ordinal  in  nature. 
It  transforms  a  strictly  increasing  sequence 

Ao  r  Ai  c  •••  c  At  ~  X’ 

of  subsets  of  X  to  an  increasing  sequence 
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where  depends  on  the  entire  input  sequence. 

Remark  3.  The  converse  of  Theorem  2  is  also  true.  To  see  tl is 
assume  that  d  has  range  hQ  <  h^  <  ...  <  ht,  that  range  F(d)  >■_  range  d, 
and  that  F(d)  depends  only  on  the  sequence 

D(hQ)  c  D(h-j )  c  ...  c  D(ht)  =  X 

of  subsets  of  X.  If  0 1  .  is  injective,  we  can  let  k.  =  f(h.) 

i dnye  u  i  i 

and  c  =  o(d) .  Then 

d(x)  <  h.  <  —  =  >  c(x)  <  ki 

so  that  Clk^  =  DCh^  for  i  =  0,1, ...,t.  But  then 
(FC)(k.)  =  (FD)(h.), 

so 

(Fc)(x)  <  ki  <  '  >  (Fd) (x)  <  hi 

and  consequently 

( Fc) (x)  =  ki  <-=->  ( Fd) (x)  =  hi. 

It  is  immediate  that 

F(od)  =  eF(d) 

as  desired. 
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In  the  most  general  situation,  a  change  of  lookup  table  invo’ves 
the  application  of  a  linear  or  piecewise  linear  mapping  o  to  L.  In 
the  linear  case,  one  is  given  non-negative  integers  a,  b  such  .hat 

0  <  a  <  b  <  255 


and  9  ,  is  defined  by  the  rule 

a  ,  D 


0a,b<h>  = 


[ 


0 

h  -  a 
b  -  a 
255 


255] 


h  £  a 

a  <  h  <  b 
h  >  b 


where  [t]  denotes  the  result  of  rounding  the  real  number  t  to  the 

nearest  integer.  When  b  =  a  +  1,  we  may  denote  0  ,  as  0  and 

d  9  D  d 

observe  that 


ea(h)  - 


h  <_  a 
h  >  a  . 


It  turns  out  that  (0,M)  -  compatibility  for  all  e  is  in  itseli  a 

d 

powerful  condition  on  a  filter  F. 


Definition.  A  filter  F  is  said  to  be  flat  if  there  is  an 
isotone  mapping  y  :  P(X)  - >  P(X)  such  that 

F(D)  =  y  o  D 

for  all  d  :  X  - ->  L. 
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Theorem  4.  Let  F  be  (e.,M)  -  compatible  for  all  t-  (l  «•  a  ^ 

a  a  - 

and  all  M  c  L.  Then  F  is  flat. 


Proof:  Let  c,  d  :  X  - >  L  be  digital  images,  and  let  a,  b  •:  L  be 

such  that  C(a)  =  D{b).  We  claim  that  (FC)(a)  =  (FD)(b).  This  is  a 
direct  consequence  of  the  fact  that 


(C  o  e+)(h) 


h  f  255 
h  =  255 


(D  o  0J)(h) 


D(b) 

, 

X 


h  f  255 
h  =  255. 


Thus  Gflc  =  O^d ,  so 

[F(C)]  o  =  [ F( D) ]  -  0+ 


and 


CF(C)](a)  =  [F(D)](b) . 

We  may  now  define  y  :  P(X) - >  P(X)  by 


Y(Y)  =  [F(D)](b) 

Y  (<t> )  =  > 

where  D(b)  =  Y  and  note  that  y  is  well  defined.  It  is  then  clear 
that 

[F(D)](h)  =  (y  o  D)(h) 


255) 


so  F  is  flat. 


Corollary  5.  If  F  is  (0.,M)  -  compatible  for  every  P_  and 

- 5  a 

every  M  c  L,  then  F  is  (0,M)  -  compatible  for  all  isotone  mappings 
0  :  L  - ■>  L  and  all  M  c  L. 

In  view  of  the  above  discussion  it  is  reasonable  to  consider  the 
theory  underlying  flat  filters.  Such  filters  are  easy  to  construct  as 
one  only  needs  to  produce  an  isotone  mapping  y  on  P(X)  such  that 
y( X)  =  X,  and  y(4>)  =  4>.  Having  recognized  this  need,  it  is  appropriate 
to  decide  upon  reasonable  conditions  for  these  mappings.  The  ideu  is 
to  attach  some  spatial  significance  as  to  whether  a  point  belongs  to  the 
output  of  the  mapping.  In  other  words,  if  Y  £  X,  the  decision  as  to 
whether  x  e  y(Y)  should  be  based  on  all  points  in  some  small  reqion 
surrounding  x.  This  may  be  precisely  stated  by  saying  that  the  map¬ 
ping  y  shall  be  point-based  in  that  for  each  x  e  X  there  is  a  subset 

N(x)  of  X  containing  x  and  a  mapping  yx  :  P(N(x))  - >  {0,T  such 

that: 


(PB1)  yx(N(x))  =  1,  and  yx(4>)  =  0. 

(PB2)  For  A  e  P(X),  x  £  y(A)  <— =  >  yx(A  n  N(x) )  =  1. 

So  that  y  will  be  independent  of  the  polarity  of  the  image,  it  'urns 
out  to  be  useful  to  have  y  preserve  complements  in  that 

(PB3)  For  A  e  P( X) ,  y( X  \  A)  =  X  \  y(A) . 

Remark  6.  For  a  point-based  isotone  mapping  y  on  P(X)  to  pre¬ 
serve  complements,  it  is  clearly  necessary  and  sufficient  that  yx  have 
the  property  that  for  every  proper  subset  A  of  N(x),  exactly  one  of 
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Now  let  y  be  point-based  with  an  associated  family  of  neighbor¬ 
hoods  N(x)(x  <  X).  If  M(x)  is  a  second  such  family,  it  is  evident 
that  so  is  M(x)  n  N(x).  In  view  of  this,  there  is  no  harm  in  assuminq 

(PB4)  If  M(x)(x  e  X)  satisfies  (PB1)  and  (PB2) ,  then  N(x)  c  M(x). 

Such  a  minimal  family  of  subsets  of  X  will  be  called  the  system  of 
neighborhoods  associated  with  y.  Unless  otherwise  specified,  when  we 
speak  of  a  point-based  isotone  mapping  y  ,  it  will  be  assumed  that 
the  family  (N(x)  :  x  <.  X)  represents  this  minimal  system  of  neighbor¬ 
hoods. 

In  order  for  y  to  interact  properly  with  planar  rigid  motions, 

we  need  to  consider  the  effect  of  translations.  These  are  mappings 

of  the  form  T  „  where  p,q  are  fixed  integers  and  T  (i,j)  =  (p  +  i ,q  +  j 

P»M  r 

Unless  p  =  0  =  q  the  domain  and  range  of  T  will  be  proper  subsets 

P  y  M 

of  X.  We  agree  to  call  the  point-based  isotone  mapping  translation  - 
invariant  provided  it  satisfies 

(PB5)  If  N(x)  is  contained  in  the  domain  of  the  translation  T, 
if  N(x)  1  {x > ,  and  if  y  =  T(x),  then  N(y)  =  T(N(x))  and 


Remark  7.  The  reason  for  the  concern  about  whether  N ( x )  =  { x > 
is  that  around  the  edge  of  an  image  there  may  not  be  enough  room  to 
form  the  desired  neighborhoods,  so  that  one  might  simply  take  N(x)  =  (x). 


Definition.  Let  y  be  a  point-based,  translation-invariant  isotone 


mapping  on  P(X).  To  say  that  y  is  frequency-defined  is  to  say  that 

there  exists  an  integer  j  such  that  if  N(x)  /  {x},  yx(A  r  N ( x ) )  =  1 

if  and  only  if  #(A  n  N(x))  £  j.  Here  #Y  denotes  the  cardinality  of 
the  set  Y. 

Theorem  8.  Let  y  satisfy  (PB1)  through  (PB5).  Let  x  be 
chosen  so  that  N(x)  f  { x)  and  let  k  =  #N(x).  Let  j  be  the  least 
cardinal  number  for  which  yx(A)  =  1  for  some  subset  A  of  N(x)  with 
j  =  #A.  Necessary  and  sufficient  conditions  for  y  to  be  frequency- 
defined  are  that  k  be  odd  and  j  =  (1  +  k)/2. 

Proof:  Let  y  be  frequency-defined.  If  #A  <  k/2,  then  #(N(x)  \  A)  > 

so  there  is  a  subset  B  of  N( x)  \  A  with  #B  =  #A.  Since  yx(B)  =  0, 
this  is  a  contradiction.  If  k  is  even,  taking  B  c M ( x )  with  ar- 
dinality  k/2  will  produce  a  similar  contradiction.  Thus  k  is 

odd  and  #A  >  k/2.  If  now  #B  =  (1  +  k)/2,  then  #B  >  #(N(x)  \  B).  It 

follows  that  y x ( B )  =  1  and  consequently  that  j  =  (1  +  k)/2. 

Suppose  conversely  that  k  is  odd  and  j  =  (1  +  k)/2.  We  must 
show  that  y x ( B )  =  1  for  all  B  £  N(x)  for  which  #B  £  j .  Since 
#B  £  j  implies  #(N(x)\B)  <  j,  it  follows  that  yx(N(x)  \  B)  =  G 
and  consequently  yx(B)  =  1. 

The  mapping  y  is  called  a  join  homomorphism  in  case 
y(S  u  T)  =  y(S)  u  y(T)  for  all  S,  T  e  P(X).  There  is  a  dual  notion 
of  meet  homomorphism,  and  to  say  that  y  is  a  homomorphism  is  to 


k/2, 
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say  that  it  is  both  a  join  and  a  meet  homomorphism.  The  next  theorem 
shows  that  these  conditions  are  extremely  powerful,  and  will  be  cnly 
met  in  trivial  circumstances. 

Theorem  9.  Let  y  be  a  point-based  isotone  mapping  on  P(X).  Then 
(la)  <==>  (lb),  (2a)  <— >  (2b)  and  (3a)  <  >  (3b). 

(la)  y  is  a  join  homomorphism. 

(lb)  For  each  x  e  X  there  is  a  subset  of  N(x)  such  that 

x  c  y(S)  <=  >  S  n  Ax  f  <j>. 

(2a)  y  is  a  meet  homomorphism. 

(2b)  For  each  x  e  X  there  is  a  subset  Bx  of  N(x)  such  that 

x  e  y(S)  <=>  S  D  Bx< 

(3a)  y  is  a  homomorphism. 

(3b)  For  each  x  there  is  an  element  yx  of  N(x)  such  tnat 
x  e  y(S)  <  >  yx  e  S. 

Proof:  (la)  -  >  (lb)  One  simply  takes  Cx  to  be  the  union  of  all 

subsets  of  N(x)  that  are  mapped  to  0  by  y  and  notes  that  >  (C  )  =  0. 

X  XX 

The  set  N(x)\Cx  then  serves  as  Ax. 

(lb)  -■>  (la)  If  x  c  Y(S  u  T),  then  (S  u  T)  n  Ax  M  .  so 

S  n  Ax  f  <j>  or  T  n  Ax  f  $.  Hence  x  e  y[S)  u  y(T). 

(2a)  =>  (2b)  One  takes  B  to  be  the  intersection  of  all 

X 

subsets  C  of  N(x)  for  which  y(C)  =  1. 

(2b)  >  (2a)  Is  obvious,  as  is  (3b)  — >  (3a). 

(3a)  (3b)  If  Bx  is  defined  as  in  the  proof  of  (2a) -  >  (2b), 

then  y  (B  )  =  1  but  yv{C)  =  0  for  all  C  c  B  .  It  is  immediate  that 
Bx  is  a  singleton  set,  and  (3b)  follows. 


I 
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Remark  10.  If  y  is  translation-invariant,  it  is  tempting  to 
say  that  if  A  is  in  the  domain  of  the  translation  T,  then 
,[T(a)]  =  T [ > ( a ) ] .  This  is  only  true  if  both  A  and  T(A)  lie  in 
that  portion  of  X  on  which  N(x)  t  { x } .  To  illustrate  this,  let 
■i  be  defined  by  3  by  3  nei ghborhouds  by  the  rule  that  >  (A1  =  1 
if  #A  >  5  for  A  €  P(N(x)).  Let  X  =  {1,2, 3, 4, 5}  x  {1,2,3,4,51, 
x  =  (2,2)  and  T  =  T^  -j.  Then  if  A  is  the  subset  indicated  by 
l's  in  Figure  2(a),  one  sees  from  Figs.  2(c)  and  2(d)  that 
TIMA)]  f  > [T ( A) 3 . 


1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

1 

1 

1 

0 

1 

1 

1 

0 

0 

0 

0 

1 

0 

0 

1 

1 

1 

0 

0 

0 

1 

1 

1 

0 

1 

1 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

A 

Fig.  2(a) 


T(  A) 

Fig.  2(b) 


y(  A) 

Fig.  2(c) 


yCt(a) ] 
Fig.  2(d) 
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$4.  Underlying  Statistical  Considerations.  Suppose  that  the 

true  input  data  d*:  X - >  L  has  been  corrupted  by  additive  or 

multiplicative  noise  so  as  to  produce  an  actual  input  d:  X  - -  t. 

One  wants  to  construct  a  flat  filter  F  that  somehow  smoothes  the 
noise  without  blurring  the  picture.  Thus  at  each  level  h,  one  wishes 
to  estimate 

A*  =  f  x  :  d*( x )  <  h } 

by  means  of  looking  at 

Ah  =  {x  :  d(x)  <  h}. 

Hopefully,  F(d)  will  provide  a  good  estimate  of  Ah  by  way  of 
{x  :  ( Fd) ( x)  <  h}. 

To  put  the  problem  in  a  specific  framework,  suppose  that  Ah  and 

★ 

A^  are  related  by  the  assumption  that  for  some  fixed  a  priori  proba¬ 
bility  p  (p  >  0.5)  one  has 

(51)  x  <_  A^  *  •-  >  x  u  A^  with  probability  p, 

(52)  x  i  A^  =•-=>  x  l  Ah  with  probability  p, 

and  that  these  probabilities  are  independent  of  the  location  of  x. 

The  goal  is  to  define  y  on  P(X)  so  that 

(1)  »  is  isotone 

(2)  y  somehow  maximizes  the  probability  of  correct  classification 
of  points.  In  other  words,  in  some  specific  sense  we  wish  to  maximize 
the  a  posterior  probability  that 


(a)  If  x  y(Ah)  then  x  A^,  and 

(b)  If  x  4  y(Ah)  then  x  /  A^. 


In  view  of  the  discussion  of  53,  it  seems  reasonable  to  restrict 
our  attention  to  translation-invariant,  point-based  isotone  mappings 
having  a  fixed  system  (N(x)  :  x  t  X}  of  neighborhoods  of  points 
Because  of  this,  we  need  only  consider  a  fixed  point  x  for  which 
N(x)  f  {x>,  and  the  mapping  y  :  P(N(x))  - >  {0,1}  defined  by 


>X(A)  =  1  <  >  x  c  y ( A ) 


for  A  N(x) . 

★ 

For  a  given  subset  Ah  of  X,  one  can  define  a  probability 

measure  p'  on  N(x)  by  the  rule  that  for  A  c  N(x),  p‘(A)  shall 

be  the  probability  of  observing  A,  given  the  true  input  data 

A^  m  N(x).  For  example,  if  N(x)  is  a  3  by  3  neighborhood  cen- 
★  • 

tered  on  x,  if  Ah  n  N(x)  and  A  are  given  by  the  matrices 


1  1  1 


0  1  1 
0  0  0 


1  0  1 


0  0  0 


0  0  1 


where  1  denotes  membership  in  a  given  set  and  0  membership  i,i  its 

5  4 

complement,  we  have  p 1 ( A)  =  p  (1  -  p)  .  A  numerical  measure  o^  the 

★ 

ability  of  >  to  distinguish  between  and  its  complement  is 
provided  by  the  gain  G(yx).  F°r  x  c  A^,  this  is  defined  to  be  the 
probability  that  for  a  randomly  chosen  subset  B  of  N(x),  >  (F)  =  1; 


added  to  the  probability  that  if  the  true  input  data  were  the  complement 
* 

of  A.,  we  would  have  y  (B)  =  0  for  B  a  randomly  chosen  subset  of 

i)  X 

N(x).  In  connection  with  this  it  is  convenient  to  let 

T  =  { B  :  B  N( x)  ,  ,(B)  =  1} 

>  *  x 

F  .  =  (B  :  B  c  N(x),  yy(B)  =  0}  . 

i  X 

It  follows  that 

G(yx)  =  E{p(A)  :Aty 

+  E{p(N(x)\A)  :  A  e  Fy). 

The  reason  for  this  is  that  if  p"  is  the  probability  measure  associ¬ 
ated  with  the  complement  of  A^,  then  for  B  <-  N(x), 

p" (B)  =  p'(N(x)  \  B) . 

One  may  now  search  for  those  mappings  y  such  that  >x  has  maximal 
gain  for  a  given  type  of  input  data.  The  search  is  aided  by  the  fol¬ 
lowing  result: 

Theorem  11.  Let  p'  be  a  probability  measure  defined  on  N(x). 
Suppose  p'  has  the  property  that 

(1)  for  A^_N(x),  p'(A)  f  p’(N(x)\  A),  and 

(2)  if  p ' (A)  >  p'(N(x)\  A}  and  if  A  <-  B  ^  N ( x ) ,  then 
P* (B)  >  p'(N(x)\  B). 


Define  >x  :  P(N(x))  - >  {0,1)  by  the  rule 


,(A)  = 


1  p‘ (A)  >  p’(N(x)  \  A) 


0  otherwise  . 


Then  is  isotone;  furthermore,  G(yx)  >_G(V^)  for  any  other  isotone 

mapping  yx - >  {0,1 }. 


Proof:  The  fact  that  ^  is  isotone  comes  from  condition  (2)  of  the 

theorem.  The  assertion  regarding  G(y')  >.  G(y'')  follows  from  an 

X  X 

examination  of  the  sets  T  , ,  Fy  ,  Ty„  and  Fy  .  Any  changes  in 
the  sums  for  G(y')  and  G(y")  must  clearly  come  from  members  of 

X  X 

(Ty.  n  Fyll )  u  (Ty,  n  Fy,). 

For  A  e  T^  n  F  „ ,  the  term  in  G(yx, )  corresponding  to  A  is  p:(A), 
while  in  G(y")  it  is  p'(N(x)\A).  By  construction  of  we 

have  p'(A)  >  p'(N(x)\A).  Similarly,  for  B  e  Ty,  n  Fy,  ,  the  term 
in  G(y' )  is  p'(N(x)\B)  while  the  one  in  G(y")  must  be  p'(B). 

Again  by  construction  of  y'  ,  p'(N(x)\  B)  >  p ' ( B )  . 


Corollary  12.  If  N(x)  c  Ah  and  k  =  #N(x)  is  odd,  then  the 
mapping  ■<,'  described  in  the  theorem  is  given  by 


y  (A)  =1  if  k/2  <  #A. 


*  *  “•>  V. 
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Remark  13.  The  mapping  in  Corollary  1 2  may  be  implemented  by 
means  of  applying  a  suitable  median  filter  to  d.  This  says  that 
within  the  framework  of  the  current  model,  the  best  way  of  smoothing 
noise  in  the  interior  is  to  use  a  median  filter. 

The  paper  will  conclude  by  carefully  examining  the  situation 

where  N(x)  is  a  3  by  3  window  centered  on  x.  The  members  of 

★ 

will  be  denoted  by  I's,  those  of  its  complement  by  0's,  and  w^ 

shall  restrict  our  attention  to  a  fixed  x.  Notice  that  the  mapping 

produced  by  Theorem  11  is  necessarily  complement  preserving.  This  is 

desirable  because  one  wants  the  output  of  a  filter  to  be  independent  jf 

the  polarity  of  the  input  data.  In  view  of  this,  we  shall  concentrate 

★  ★ 

on  input  data  for  which  x  r  and  assume  that  5  <_  #(N(x)  n  A^).  We 
shall  begin  by  looking  at  5  types  of  input  data.  They  will  be  denoted 
type  i  (i  =  0/1,2, 3, 4),  according  to  whether  or  not  N(x)  has 

•k 

i  =  0,1, 2, 3,  or  4  members  of  X\A^.  Before  doing  the  theoretical 
analysis,  let  us  examine  some  calculated  probabilities  of  correct  classifi¬ 
cations.  This  is  done  in  Table  3.  There  are  two  items  of  interest  h?re. 
For  the  type  4  case,  the  filters  all  seem  to  decrease  the  probability  of 
correct  classification.  This  is  hardly  surprising  since  in  this  instance 

the  8  immediate  neighbors  cf  the  given  pixel  all  have  equal  chances  of 

★ 

representing  members  of  Ah  as  opposed  to  its  complement.  Thus  no 
additional  information  can  be  obtained  by  looking  at  the  8  neighbors  of 
x.  The  second  observation  is  that  in  the  type  3  case  the  answer  seems 
to  depend  on  the  numerical  value  p  of  the  a  priori  probability  of 
correct  classification. 


One  can  briny  Theorem  11  lo  bear  on  the  j'roblem  by  simply  cal* 

9 

culating  probabilities  of  occurrence  of  each  of  the  2  possible 
subsets  of  N(x),  given  the  a  priori  assumption  regarding  the  nature 
of  ii  N(x).  This  has  been  done  but  the  rather  tedious  details 
will  not  be  repeated  here.  Rather,  we  shall  simply  list  the  results 
in  Table  4. 


i  DATA  |  Best  Isotone 
TYPE  j  Method 

0  j  of  Table  3 


+ 


1 

>A  of 

Table 

3 

2 

' - 1 

'A  of 

Table 

!  ^ 

1 

j  There  is  a  critical  value  Pg  of  p(.8958  Pg  <  .9090)  1 

|  such  that  for  p  pQ  ,  is  the  best  method  and  for 

I  1 

p  <  Pg,  of  Table  3  is  best.  pQ  is  determined  ty 

solving  the  equation  p  =  x ( 1  -  p)  where  x  is  the  "oot  • 
j  of  the  equation  -  9x^  +  6x^  -  24x^  +  6x‘  -  4x  +  1  *  0  ; 

lying  between  x  =  8.6  and  x  =  9.  ! 

|  x  •  »(Ah)  if  x  Ah  . 

| 


Table  4.  Summary  of  "best"  flat  filters  based  on  3  by  3 
nei ghborhoods . 
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Up  to  this  point  it  has  been  assumed  that  the  center  pixel  x 

★  ★ 
was  a  member  of  ,  and  that  any  other  members  of  N(x)  Af 

were  randomly  scattered  through  N(x)  \  fxL  The  situation  changes 

dramatically  if  more  information  is  known.  To  see  this,  note  that 

N(x)  consists  of  three  data  types  as  indicated  below: 

c  b  c 

b  a  b 

c  b  c 

The  center  pixel  is  type  a,  its  North,  South,  East,  West  neighbor., 

are  type  b,  and  the  remaining  four  corner  pixels  are  type  c.  0ur 

★ 

basic  assumption  will  now  be  that  x  .  Ah,  and  that  we  know  the  num¬ 
ber  of  type  b  as  well  as  the  number  of  type  c  pixels  that  are  in  the 

★ 

complement  of  A^,  but  not  their  precise  location.  Thus  a  data  type 

may  be  specified  by  an  ordered  pair  of  integers  that  denotes  the 

number  of  pixels  of  types  b  and  c  that  lie  in  the  complement  of 

★ 

A^.  For  example,  data  type  (1,2)  indicates  the  presence  of  one  type 

★ 

b  pixel  and  two  type  c  pixels  in  the  complement  of  A^.  The  cal¬ 
culations  inovlved  in  applying  Theorem  11  are  again  tedious,  so  they 
will  be  omitted.  By  way  of  illustration,  however,  the  type  (0,1)  - 
case  will  be  considered  in  some  detail. 

Recall  that  p  denotes  the  a  priori  probability  of  the  input 
data  being  correct,  and  set  q  =  1  -  p.  The  probability  of  observing 
variour  numbers  of  Vs  in  the  corners  is  then  seen  to  be 
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DATA 

TYPE 

A 

N  ( x )  \  A 

P*  (A) 

(5,0) 

(0,4) 

I  6  3 

•  p  q 

(4,0 

0,4) 

5  4 

•  p  q 

(3,0) 

(2,4) 

4  5 

p  q 

(5,1) 

(0,3) 

5  4  A  -  7  2 
*  P  q  +  3p  q 

(4,1) 

(1,3) 

4  5  .  -  6  3 
•  P  q  +  3p  q 

(3,1) 

(2.3) 

,  5  4  3  6 

3p  q  +  P  q 

NOTE:  The  dots  indicate  the  higher  of  the  two  probabilities,  in 
each  row. 

The  actual  results  are  summarized  in  Table  5. 


DATA  TYPE 

BEST  ISOTONE  METHOD  ,x(A)  -  1  if 

(0,1) 

6  ■  //N  (x)  n  A  or 

5  =  #N(x)  H  A  unless  N(x)  n  A  is  type  (1,4)  or 

4  =  $N(x)  -  A  provided  N(x)  ■  A  is  type  (4,0) 

(0,2) 

A  ,i  N(x)  has  at  least  3  members  that  are  type  a  or 

_  . 

1 

(1,2) 

Same  as  (0,2) 

(1,1) 

6  <  #N(x)  u  A  or 

5  =  #N(x)  i.  A  unless  N(x)  A  is  type  (0,1,4)  or 

(0,4,1) 

4  =  #N(x)  n  A  provided  N(x)  n  A  is  of  type  (1,3,0) 

or  (1,0,3). 

NOTE:  The  types  (1,0),  (2,0)  and  (2,1)  are  handled  by  symmetry 
Table 5.  Summary  of  "best"  flat  filters  based  on  3  by  3 


nei qhborhoods . 


As  a  final  item,  let  us  see  how  well  the  "best"  flat  filters  do  w  th 
various  types  of  data. 


Table  J>.  A  posterieri  probability  of  correct  classification  for 
data  of  given  type  using  "best"  filter  method  as  speci 
fied  in  Theorem  1 1 . 
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